<html>
<head><meta charset="utf-8"><title>Design meeting: syntax trees · t-compiler/rust-analyzer · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/index.html">t-compiler/rust-analyzer</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html">Design meeting: syntax trees</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="158793436"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158793436" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158793436">(Feb 18 2019 at 11:37)</a>:</h4>
<p>The meeting will happen at Wednesday, 20th, 14:00 -- 15:00 UTC, in this Zulip stream. </p>
<p>The goal of the meeting is to nail down the API of syntax trees for rls 2.0/compiler.</p>
<p>Required reading:</p>
<ul>
<li>
<p><a href="https://github.com/apple/swift/tree/master/lib/Syntax" target="_blank" title="https://github.com/apple/swift/tree/master/lib/Syntax">Swift libsyntax</a> -- the design of full-fidelity syntax trees I'd like to use for rls-2.0</p>
</li>
<li>
<p><a href="https://github.com/rust-analyzer/rowan" target="_blank" title="https://github.com/rust-analyzer/rowan">rowan</a> -- rust library I use for implementing syntax trees.</p>
</li>
<li>
<p><a href="https://github.com/rust-analyzer/rust-analyzer/blob/bd2b4ef2d1c631215523f79a8138cfa8cbd3f70e/crates/ra_syntax/src/syntax_node.rs" target="_blank" title="https://github.com/rust-analyzer/rust-analyzer/blob/bd2b4ef2d1c631215523f79a8138cfa8cbd3f70e/crates/ra_syntax/src/syntax_node.rs">raw</a> and <a href="https://github.com/rust-analyzer/rust-analyzer/blob/bd2b4ef2d1c631215523f79a8138cfa8cbd3f70e/crates/ra_syntax/src/ast.rs" target="_blank" title="https://github.com/rust-analyzer/rust-analyzer/blob/bd2b4ef2d1c631215523f79a8138cfa8cbd3f70e/crates/ra_syntax/src/ast.rs">typed</a> syntax tree API which exist in rust-analyzer today.</p>
</li>
</ul>
<p>rowan/SyntaxNode is pretty close to Swift libsyntax, the two main differences are:</p>
<ul>
<li>to get the specific node (say, struct's name), we iterate children instead of getting the nth child directly</li>
<li>"trivia" nodes are just usual nodes in the tree, they are not attached to tokens.</li>
</ul>



<a name="158972444"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158972444" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158972444">(Feb 20 2019 at 13:05)</a>:</h4>
<p><span class="user-group-mention" data-user-group-id="1060">@WG-rls2.0</span> reminder , meeting starts in one hour!</p>



<a name="158976748"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158976748" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158976748">(Feb 20 2019 at 14:02)</a>:</h4>
<p><span class="user-group-mention" data-user-group-id="1060">@WG-rls2.0</span> meeting time!</p>



<a name="158976817"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158976817" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158976817">(Feb 20 2019 at 14:02)</a>:</h4>
<p>Let's see who is here</p>



<a name="158976823"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158976823" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158976823">(Feb 20 2019 at 14:02)</a>:</h4>
<p>/me</p>



<a name="158976825"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158976825" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158976825">(Feb 20 2019 at 14:02)</a>:</h4>
<p>/me admits they have not done the homework. <span aria-label="embarrassed" class="emoji emoji-1f633" role="img" title="embarrassed">:embarrassed:</span></p>



<a name="158976827"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158976827" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158976827">(Feb 20 2019 at 14:02)</a>:</h4>
<p>o/</p>



<a name="158976870"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158976870" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158976870">(Feb 20 2019 at 14:03)</a>:</h4>
<p>I'm here, might not be 100% present though and I don't know how much I can contribute on this topic ;)</p>



<a name="158976945"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158976945" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158976945">(Feb 20 2019 at 14:03)</a>:</h4>
<p><span aria-label="wave" class="emoji emoji-1f44b" role="img" title="wave">:wave:</span></p>



<a name="158977069"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977069" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977069">(Feb 20 2019 at 14:04)</a>:</h4>
<p>Ok, so let's get started!</p>



<a name="158977078"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977078" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977078">(Feb 20 2019 at 14:05)</a>:</h4>
<p>We should also invite <span class="user-mention" data-user-id="119009">@eddyb</span> in case they don't know about this meeting yet</p>



<a name="158977267"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977267" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977267">(Feb 20 2019 at 14:06)</a>:</h4>
<p>Pinged the whole of WG-grammar on discord just in case</p>



<a name="158977278"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977278" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977278">(Feb 20 2019 at 14:06)</a>:</h4>
<p><span class="user-mention" data-user-id="133169">@matklad</span> , based on my cursory review of Swift lib/Syntax vs rowan, my first question is: Rowan seems to be abstracted over what concrete syntax the client might want</p>



<a name="158977303"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977303" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977303">(Feb 20 2019 at 14:07)</a>:</h4>
<p>Yep!</p>



<a name="158977304"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977304" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977304">(Feb 20 2019 at 14:07)</a>:</h4>
<p>is lib/Syntax similar? Or does it hard-code knowledge of Swift syntax itself in that library?</p>



<a name="158977316"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977316" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977316">(Feb 20 2019 at 14:07)</a>:</h4>
<p>It's even slightly more subtle than that</p>



<a name="158977347"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977347" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977347">(Feb 20 2019 at 14:07)</a>:</h4>
<p>both rowan and swift libsyntax use a single type as the underlying storage for all nodes</p>



<a name="158977422"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977422" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977422">(Feb 20 2019 at 14:08)</a>:</h4>
<p>Specifically, swift uses this "DST" type to store any node: <a href="https://github.com/apple/swift/blob/e066d452e8128d9ac5d457d6c6a81d67e850541d/include/swift/Syntax/RawSyntax.h#L221-L223" target="_blank" title="https://github.com/apple/swift/blob/e066d452e8128d9ac5d457d6c6a81d67e850541d/include/swift/Syntax/RawSyntax.h#L221-L223">https://github.com/apple/swift/blob/e066d452e8128d9ac5d457d6c6a81d67e850541d/include/swift/Syntax/RawSyntax.h#L221-L223</a></p>



<a name="158977458"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977458" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977458">(Feb 20 2019 at 14:09)</a>:</h4>
<p>It looks like this in memory:</p>
<div class="codehilite"><pre><span></span>kind|n_children|child1|child2|child3
</pre></div>



<a name="158977483"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977483" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977483">(Feb 20 2019 at 14:09)</a>:</h4>
<p>where <code>kind</code> is an <code>u16</code> tag for the specific node kind (Class Declaration, sum expression etc)</p>



<a name="158977594"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977594" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977594">(Feb 20 2019 at 14:11)</a>:</h4>
<p>The <code>concrete syntax the client might want</code> is then a new-type wrapper around this node, which adds static info like "the first child is expected to be an identifier".</p>



<a name="158977643"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977643" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977643">(Feb 20 2019 at 14:11)</a>:</h4>
<p>does this make sense?</p>



<a name="158977657"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977657" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977657">(Feb 20 2019 at 14:12)</a>:</h4>
<p>i think so</p>



<a name="158977809"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977809" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977809">(Feb 20 2019 at 14:13)</a>:</h4>
<p>Great!</p>
<p>So, the primary question is "would a similar implementation work everywhere in the compiler?".</p>
<p>I defenitelly like it for IDE, for the following reasons:</p>
<ul>
<li>you have comments &amp; whitespace in children</li>
<li>you can get to each node's parent</li>
<li>there's a "common super type" of all syntax nodes, so you can traverse the tree as you like</li>
</ul>



<a name="158977904"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977904" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977904">(Feb 20 2019 at 14:14)</a>:</h4>
<p>Note that this impl is not really customizable: if you want to associated additional info with syntax nodes, you'll have to use some sort of a side-table.</p>



<a name="158977932"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977932" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jeremy Kolb <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977932">(Feb 20 2019 at 14:15)</a>:</h4>
<p>Does Swift use this approach in their compiler as well or just IDE? I guess why wouldn't it work?</p>



<a name="158977963"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977963" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977963">(Feb 20 2019 at 14:15)</a>:</h4>
<p>from the point of incremental compilation you usually want to restrict the amount of data that can be accessed without tracking</p>



<a name="158977970"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158977970" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158977970">(Feb 20 2019 at 14:15)</a>:</h4>
<p>IIRC, libsytax is a new thing, and they are in the process of transitioning compiler to it.</p>



<a name="158978022"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978022" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978022">(Feb 20 2019 at 14:16)</a>:</h4>
<p>i.e. if you get the parent of a node, it should go through a query</p>



<a name="158978049"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978049" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978049">(Feb 20 2019 at 14:16)</a>:</h4>
<p>One note about comments/whitespace: it feels to me like such things should be...recoverable, but not necessarily something we want 99% of the time.</p>



<a name="158978077"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978077" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978077">(Feb 20 2019 at 14:16)</a>:</h4>
<blockquote>
<p>Note that this impl is not really customizable: if you want to associated additional info with syntax nodes, you'll have to use some sort of a side-table.</p>
</blockquote>
<p>But this part feels ok to me</p>



<a name="158978120"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978120" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978120">(Feb 20 2019 at 14:17)</a>:</h4>
<p>most queries probably don't directly access the syntax, but an intermediate representation  generated from it by another query?</p>



<a name="158978123"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978123" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978123">(Feb 20 2019 at 14:17)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span>  true, yeah. I expect that perhaps you might not want to use syntax trees <strong>directly</strong> at all. syntax node remembers absolute offset , and that's bad for incremental</p>



<a name="158978160"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978160" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978160">(Feb 20 2019 at 14:17)</a>:</h4>
<p>so yeah, what <span class="user-mention" data-user-id="129457">@Florian Diebold</span> , there could be an intermediate lowred representation (<strong>A</strong>ST)</p>



<a name="158978181"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978181" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978181">(Feb 20 2019 at 14:18)</a>:</h4>
<p>yes, some sort of reduced representation for specific purposes would solve that problem</p>



<a name="158978267"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978267" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978267">(Feb 20 2019 at 14:18)</a>:</h4>
<p>I'm not entirely convinced. It depends on what the minimum amount of work you want to do is</p>



<a name="158978271"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978271" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978271">(Feb 20 2019 at 14:18)</a>:</h4>
<p>re "recoverability", the biggest drawback of such trees is memory footprint. like, every token like <code>+</code> is a separate allocation.</p>



<a name="158978302"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978302" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978302">(Feb 20 2019 at 14:19)</a>:</h4>
<p>in particular, if you have your base representation with absolute offsets etc, you will have to recompute the entire tree for each parse</p>



<a name="158978305"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978305" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978305">(Feb 20 2019 at 14:19)</a>:</h4>
<p>maybe that's ok</p>



<a name="158978309"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978309" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978309">(Feb 20 2019 at 14:19)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> when you say comments+whitespace should be recoverable, are you saying you object to having <del>them always</del> their representation embedded in parse tree? even if e.g. we did not guarantee their presence?</p>



<a name="158978315"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978315" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978315">(Feb 20 2019 at 14:19)</a>:</h4>
<p>also, memory allocation</p>



<a name="158978337"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978337" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978337">(Feb 20 2019 at 14:19)</a>:</h4>
<p>could we have two parsers: one that immediately produces the reduced form and another one that is invoked only when needed?</p>



<a name="158978449"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978449" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978449">(Feb 20 2019 at 14:20)</a>:</h4>
<p>the fast one wouldn't need to be error tolerant even? not sure</p>



<a name="158978450"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978450" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978450">(Feb 20 2019 at 14:20)</a>:</h4>
<p><span class="user-mention" data-user-id="116083">@pnkfelix</span> I'm not saying I <em>object</em>, but I'm not sure it's a good idea. I think that maybe an initial parse should be something that is relatively incremental friendly and contains the information we want most often, and perhaps there is a follow-on query that gives a more detailed tree when needed (as <span class="user-mention" data-user-id="124287">@mw</span> sort of suggested).</p>



<a name="158978470"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978470" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978470">(Feb 20 2019 at 14:20)</a>:</h4>
<p>In other words, the right setup might not be to have a "maximal" tree at the "base"</p>



<a name="158978473"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978473" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978473">(Feb 20 2019 at 14:20)</a>:</h4>
<p>^ I like the approach for fast parsing and then lazily geting more details when necessary</p>



<a name="158978524"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978524" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978524">(Feb 20 2019 at 14:21)</a>:</h4>
<blockquote>
<p>the fast one wouldn't need to be error tolerant even? not sure</p>
</blockquote>
<p>it does still need to be error tolerant -- we want to e.g. still run type inference on incomplete code</p>



<a name="158978543"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978543" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978543">(Feb 20 2019 at 14:21)</a>:</h4>
<p>I guess that would be possible, but with non-trivial accidental complexity.</p>



<a name="158978552"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978552" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978552">(Feb 20 2019 at 14:21)</a>:</h4>
<p><span class="user-mention" data-user-id="129457">@Florian Diebold</span> yes, that's why I was hesitant about that assumption</p>



<a name="158978554"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978554" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978554">(Feb 20 2019 at 14:21)</a>:</h4>
<p>I wonder how much to we <strong>need</strong> to optimize syntax trees?</p>



<a name="158978639"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978639" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978639">(Feb 20 2019 at 14:22)</a>:</h4>
<p>I mean, if we through them away before typechecking, they won't affect high water mark memory usage</p>



<a name="158978640"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978640" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978640">(Feb 20 2019 at 14:22)</a>:</h4>
<p><span class="user-mention" data-user-id="133169">@matklad</span> you said that the full syntax tree uses a lot of memory. is it also expensive to generate?</p>



<a name="158978698"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978698" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978698">(Feb 20 2019 at 14:23)</a>:</h4>
<p>I agree that it would make things more complicated</p>



<a name="158978714"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978714" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978714">(Feb 20 2019 at 14:23)</a>:</h4>
<p>The last time I've done measurements, I think ra_syntax parser was about 2x slower than current libsyntax, and most slowness was constructing the data structure .</p>



<a name="158978751"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978751" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978751">(Feb 20 2019 at 14:24)</a>:</h4>
<p>we could try to generate both parsers from a single grammar definition</p>



<a name="158978803"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978803" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978803">(Feb 20 2019 at 14:24)</a>:</h4>
<p>not sure how much manual intervention error correction needs</p>



<a name="158978831"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978831" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978831">(Feb 20 2019 at 14:24)</a>:</h4>
<p>well, I think we still want a single data type for the syntax tree?</p>



<a name="158978872"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978872" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978872">(Feb 20 2019 at 14:25)</a>:</h4>
<p>Or do we? Could we directly produced a lowered representation?</p>



<a name="158978972"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978972" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978972">(Feb 20 2019 at 14:26)</a>:</h4>
<p>For "lowered represenation", I was thinking about something more ECS-style, where there's no real tree, but things like <code>FunctionId</code>, <code>StructId</code>, etc,</p>



<a name="158978978"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158978978" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158978978">(Feb 20 2019 at 14:26)</a>:</h4>
<p>it should be possible to directly produce something as abstract as the compiler's current libsyntax</p>



<a name="158979007"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979007" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979007">(Feb 20 2019 at 14:26)</a>:</h4>
<p>In the ID setup, lazyly lowering function body should be doable (In fact, rust-analzyer does something like this now)</p>



<a name="158979009"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979009" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979009">(Feb 20 2019 at 14:26)</a>:</h4>
<p>that should be possible too, I think?</p>



<a name="158979065"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979065" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979065">(Feb 20 2019 at 14:27)</a>:</h4>
<p>for the "batch" compiler case we only need the "lowered" version unless there's an error, right?</p>



<a name="158979095"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979095" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979095">(Feb 20 2019 at 14:27)</a>:</h4>
<p>@matklad the "lowered representation" you mention is the equivalent of our current <code>DefId</code> setup that we end up with post-name resolution, right?</p>



<a name="158979100"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979100" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979100">(Feb 20 2019 at 14:27)</a>:</h4>
<blockquote>
<p>it should be possible to directly produce something as abstract as the compiler's current libsyntax</p>
</blockquote>
<p>Yeah, but than I think you'll end up in situation where you have to duplicate code between "full" and "reduced" syntax trees, if they are different type.</p>



<a name="158979168"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979168" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979168">(Feb 20 2019 at 14:28)</a>:</h4>
<blockquote>
<p>that we end up with post-name resolution, right?</p>
</blockquote>
<p>I'd say post macro-expansion</p>



<a name="158979194"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979194" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979194">(Feb 20 2019 at 14:28)</a>:</h4>
<p>also about tokens for macros, the lossless format would allow us to easily create those from the CST, right?</p>



<a name="158979250"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979250" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979250">(Feb 20 2019 at 14:29)</a>:</h4>
<p>I'd love to be able to end up with incrementally-stable entity ids after just parsing but not sure how much macro expansion will throw wrench into that</p>



<a name="158979268"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979268" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979268">(Feb 20 2019 at 14:29)</a>:</h4>
<p><span class="user-mention" data-user-id="153740">@Igor Matuszewski</span> it should be easy to recover tokens themselves. Recovering hygiene info would be less direct, because there wouldn't be hygiene directly in the tree.</p>



<a name="158979304"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979304" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979304">(Feb 20 2019 at 14:29)</a>:</h4>
<p>right</p>



<a name="158979415"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979415" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979415">(Feb 20 2019 at 14:30)</a>:</h4>
<p>(hope I didn't mix multiple concerns there)</p>



<a name="158979434"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979434" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979434">(Feb 20 2019 at 14:30)</a>:</h4>
<p>Can I step back a second and ask a meta question? I'm feeling a <em>touch</em> confused about the goals for this meeting but also the overall place that these trees fit into the plan. I'm wondering if a good goal for the meeting -- rather than settling on a <em>fixed</em> design -- would be to generate one or two basic options and have people try to elaborate those into more detailed proposals, so that we could then meet again to discuss them in more depth?</p>



<a name="158979468"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979468" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979468">(Feb 20 2019 at 14:31)</a>:</h4>
<p>Or maybe we'll centralize on one approach, not sure.</p>



<a name="158979477"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979477" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979477">(Feb 20 2019 at 14:31)</a>:</h4>
<p>I'm feeling a bit confused about all the considerations</p>



<a name="158979479"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979479" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979479">(Feb 20 2019 at 14:31)</a>:</h4>
<p>do we have our constraints listed somewhere?</p>



<a name="158979525"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979525" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979525">(Feb 20 2019 at 14:31)</a>:</h4>
<p>Good question!</p>



<a name="158979534"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979534" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979534">(Feb 20 2019 at 14:31)</a>:</h4>
<p>a list of constraints might be a good intermediate goal</p>



<a name="158979591"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979591" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979591">(Feb 20 2019 at 14:32)</a>:</h4>
<p>/me is not even sure what the exhaustive list of requirements is.</p>



<a name="158979632"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979632" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979632">(Feb 20 2019 at 14:32)</a>:</h4>
<p>(but that might have been elaborated at some all hands meeting that I’ve forgot about/missed)</p>



<a name="158979651"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979651" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979651">(Feb 20 2019 at 14:33)</a>:</h4>
<p>I would <strong>love</strong> to get the general idea about design sooner rather than later: if we are sure how trees should look like, we can do fun things like port rustft, etc...</p>



<a name="158979685"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979685" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979685">(Feb 20 2019 at 14:33)</a>:</h4>
<p>for constrains, the list on swift libsyntax looks good</p>



<a name="158979750"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979750" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979750">(Feb 20 2019 at 14:34)</a>:</h4>
<p>can you reproduce it here or put a direct link?</p>



<a name="158979766"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979766" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979766">(Feb 20 2019 at 14:34)</a>:</h4>
<p>this? <a href="https://github.com/apple/swift/tree/master/lib/Syntax#design-and-implementation-guidelines" target="_blank" title="https://github.com/apple/swift/tree/master/lib/Syntax#design-and-implementation-guidelines">https://github.com/apple/swift/tree/master/lib/Syntax#design-and-implementation-guidelines</a></p>



<a name="158979767"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979767" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979767">(Feb 20 2019 at 14:34)</a>:</h4>
<p><a href="https://github.com/apple/swift/tree/master/lib/Syntax#design-and-implementation-guidelines" target="_blank" title="https://github.com/apple/swift/tree/master/lib/Syntax#design-and-implementation-guidelines">https://github.com/apple/swift/tree/master/lib/Syntax#design-and-implementation-guidelines</a></p>



<a name="158979770"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979770" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979770">(Feb 20 2019 at 14:34)</a>:</h4>
<p>yep</p>



<a name="158979819"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979819" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979819">(Feb 20 2019 at 14:35)</a>:</h4>
<p>the "full-fidelity" bit is defenitlly a must-have for an IDE</p>



<a name="158979834"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979834" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979834">(Feb 20 2019 at 14:35)</a>:</h4>
<p>but what does that mean inthe presence of macros ?</p>



<a name="158979859"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979859" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979859">(Feb 20 2019 at 14:35)</a>:</h4>
<p>is the assumption there you're only talking about printing pre-expansion? We want the parse tree to be able to represent both pre- and post-expanded state</p>



<a name="158979921"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979921" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979921">(Feb 20 2019 at 14:36)</a>:</h4>
<p>(i'm looking at: ""full fidelity" - parsing a source file and printing the syntax tree should result in the same file.")</p>



<a name="158979956"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979956" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979956">(Feb 20 2019 at 14:36)</a>:</h4>
<p>Yeah, that defenitlly should hold for code typed-in by the user</p>



<a name="158979960"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979960" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979960">(Feb 20 2019 at 14:36)</a>:</h4>
<p>but if the main point is that one should always be able to <em>recover</em> something identical to the original, then fine</p>



<a name="158979975"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158979975" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158979975">(Feb 20 2019 at 14:36)</a>:</h4>
<p>in a similar vain: do we need to do macro expansion on the full fidelity tree?</p>



<a name="158980001"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980001" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980001">(Feb 20 2019 at 14:37)</a>:</h4>
<p>That probably means that macros should not modify tree in-place, but rather record that a specific macro expansion expands to a new tree</p>



<a name="158980014"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980014" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980014">(Feb 20 2019 at 14:37)</a>:</h4>
<p>(But the implication would then be that there should be enough information post-expansion to recover the pre-expanded form)</p>



<a name="158980034"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980034" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980034">(Feb 20 2019 at 14:37)</a>:</h4>
<p>or that: that we wouldn't modify tree in place</p>



<a name="158980055"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980055" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980055">(Feb 20 2019 at 14:37)</a>:</h4>
<p>That is, IDE should be able to get both the syntax tree represneting the token tree of macro invocation, and the syntax tree representing the <strong>result</strong> of invocation</p>



<a name="158980268"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980268" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980268">(Feb 20 2019 at 14:39)</a>:</h4>
<p>Back to requirements list:</p>
<p>I do want a homogeneous traversal API, with parent, first_child, next_sibling methods.</p>



<a name="158980403"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980403" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980403">(Feb 20 2019 at 14:40)</a>:</h4>
<p>In IDE contexts, such API are useful because you typically start at the token at the cursor and need to traverse the tree <strong>bottom up</strong> to do useful stuff. This is in contrast to usual compiler functionality, where you traverse the tree <strong>top down</strong>.</p>



<a name="158980439"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980439" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980439">(Feb 20 2019 at 14:41)</a>:</h4>
<p>I think being able to walk a tree of syntax without having to "know" the types deeply is a good constraint</p>



<a name="158980470"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980470" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980470">(Feb 20 2019 at 14:41)</a>:</h4>
<p>It's not entirely clear to me whether the tree should be a data structure you get back from a single query, or whether it should be something you explore <em>via</em> queries</p>



<a name="158980492"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980492" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980492">(Feb 20 2019 at 14:41)</a>:</h4>
<p>So basically a parent pointer, which in this case has to be of a universal dynamic node type?</p>



<a name="158980497"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980497" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980497">(Feb 20 2019 at 14:41)</a>:</h4>
<p>I think this <span class="user-mention" data-user-id="133169">@matklad</span> is sort of what you meant by "ECS-like" earlier?</p>



<a name="158980546"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980546" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980546">(Feb 20 2019 at 14:42)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> yeah</p>



<a name="158980559"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980559" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980559">(Feb 20 2019 at 14:42)</a>:</h4>
<p>e.g., if you had some kind of "syntax node id", you might be able to ask for things like "give me the parent of this id"</p>



<a name="158980644"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980644" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980644">(Feb 20 2019 at 14:43)</a>:</h4>
<p><span class="user-mention" data-user-id="133169">@matklad</span> one thing I personally am not very clear on is what rust-analysis is currently using and what problems you see with it</p>



<a name="158980657"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980657" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980657">(Feb 20 2019 at 14:43)</a>:</h4>
<p>part of this is that I've been too busy to do my homework</p>



<a name="158980662"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980662" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980662">(Feb 20 2019 at 14:43)</a>:</h4>
<p>This assumes salsa-style incremental parsing, right?</p>



<a name="158980693"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980693" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980693">(Feb 20 2019 at 14:43)</a>:</h4>
<p>If we use absolute offsets for each node/id, how do we guarantee stability across multiple reparses (so that we don't thrash the entire incremental cache if ids change)?</p>



<a name="158980697"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980697" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980697">(Feb 20 2019 at 14:44)</a>:</h4>
<p>not necessarily, I mean the way I would probably handle this in salsa is to have some initial function that does the parse, producing a result, but expect most users to interact with it through queries that pull out bits of it</p>



<a name="158980740"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980740" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980740">(Feb 20 2019 at 14:44)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> ra does the same thing as Swift's new syntax library (which is the same thing as Roslyn's desing)</p>



<a name="158980750"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980750" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980750">(Feb 20 2019 at 14:44)</a>:</h4>
<p>if we want things to be incremental it kind of has to go through queries, right?</p>



<a name="158980765"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980765" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980765">(Feb 20 2019 at 14:44)</a>:</h4>
<p>well, the implementation detail differ, but the general structure is the same.</p>



<a name="158980781"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980781" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980781">(Feb 20 2019 at 14:44)</a>:</h4>
<p>That is, I do not use queries for parsing.</p>



<a name="158980808"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980808" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980808">(Feb 20 2019 at 14:44)</a>:</h4>
<p>that general structure being:</p>
<ul>
<li>a lower layer that contains all the details (Concrete Syntax Tree, so to speak)</li>
<li>higher layers that abstract these into ASTs</li>
</ul>



<a name="158980824"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980824" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980824">(Feb 20 2019 at 14:45)</a>:</h4>
<p>?</p>



<a name="158980830"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980830" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980830">(Feb 20 2019 at 14:45)</a>:</h4>
<p>Yeah</p>



<a name="158980838"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980838" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980838">(Feb 20 2019 at 14:45)</a>:</h4>
<p>so a separate pass that can be incremental on its own and produces a stable-referencable(?) IDs/nodes?</p>



<a name="158980846"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980846" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980846">(Feb 20 2019 at 14:45)</a>:</h4>
<p>It seems like the macro expansion is also more at the AST level, right?</p>



<a name="158980888"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980888" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980888">(Feb 20 2019 at 14:46)</a>:</h4>
<p>By this I mean: you want to be able to get an AST with the results of macro expansion eventually. But you don't necessarily expect to get a <em>concrete syntax tree</em> with said results.</p>



<a name="158980939"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980939" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980939">(Feb 20 2019 at 14:46)</a>:</h4>
<p>Of course, you'd like to be able to serialize it back out perhaps for debugging, refactoring, etc (I can imagine a "expand macro" feature in an IDE, for example)</p>



<a name="158980962"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980962" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980962">(Feb 20 2019 at 14:46)</a>:</h4>
<p><span class="user-mention" data-user-id="116009">@nikomatsakis</span> sort-of: macro expansion works on tokens, which are close to CST than to AST</p>



<a name="158980972"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980972" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980972">(Feb 20 2019 at 14:46)</a>:</h4>
<p>The output of macro expansion needs to  be parsable, tho</p>



<a name="158980977"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980977" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980977">(Feb 20 2019 at 14:46)</a>:</h4>
<p>It would make sense for macro expansion to occur as <em>part</em> of lowering into the AST</p>



<a name="158980993"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158980993" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158980993">(Feb 20 2019 at 14:47)</a>:</h4>
<p>that is, I imagine AST does not really cares about trailing commas, but macros do</p>



<a name="158981003"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981003" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981003">(Feb 20 2019 at 14:47)</a>:</h4>
<p>I know in some cases today the output is not, so that's something to have in mind</p>



<a name="158981029"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981029" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981029">(Feb 20 2019 at 14:47)</a>:</h4>
<p>so that you have the whatever libsyntax provides initially as a holistic representation of source and AST as the macro-expanded form.</p>



<a name="158981124"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981124" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981124">(Feb 20 2019 at 14:48)</a>:</h4>
<p>Well, <span class="user-mention" data-user-id="133169">@matklad</span>, I imagine that AST notes may be "unexpanded macro", which contains tokens, but once they are expanded, they are parsed. And moreover in order to even <em>do</em> macro expansion, we need to do name resolution, which presumably wants to be based on the AST, right?</p>



<a name="158981221"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981221" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981221">(Feb 20 2019 at 14:49)</a>:</h4>
<p>I'm still not clear on the interaction between macro expansion an name resolution</p>



<a name="158981263"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981263" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981263">(Feb 20 2019 at 14:50)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span> macro expansion can introduce new names into the scope which then influence expansion of the macros elsewhere(?)</p>



<a name="158981297"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981297" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981297">(Feb 20 2019 at 14:50)</a>:</h4>
<p>does it need the full name resolution logic or some rather reduced subset?</p>



<a name="158981318"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981318" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981318">(Feb 20 2019 at 14:50)</a>:</h4>
<p><span class="user-mention" data-user-id="124287">@mw</span> I'm not sure exactly the sense of your question, but basic idea is that:</p>
<p>to expand <code>foo::bar!</code>, you have to resolve the path <code>foo::bar</code></p>



<a name="158981345"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981345" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981345">(Feb 20 2019 at 14:50)</a>:</h4>
<p>And, of course ,that may generate new names that affect later paths.</p>



<a name="158981348"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981348" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981348">(Feb 20 2019 at 14:50)</a>:</h4>
<p>are macros hygienic?</p>



<a name="158981358"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981358" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981358">(Feb 20 2019 at 14:50)</a>:</h4>
<p>right, but I don't think we should directly produce AST from the result of macro expansion. I imagine the flow where macro expands to a token-tree, which is then passed to a parser that produces a "full-fidelity" tree (with dummy whitespace) whihc is then feeded back into AST-lowering procedure</p>



<a name="158981404"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981404" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nagisa <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981404">(Feb 20 2019 at 14:51)</a>:</h4>
<p>from the technical standpoint name resolution can be done as part of the same lowering/expansion procedure. We would need multiple passes, but that is something we need today anyway.</p>



<a name="158981421"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981421" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981421">(Feb 20 2019 at 14:51)</a>:</h4>
<blockquote>
<p>are macros hygienic?</p>
</blockquote>
<p>Today, the answer is "sort of" -- not at the item level, in particular. But maybe worth asking why you're asking =)</p>



<a name="158981441"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981441" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981441">(Feb 20 2019 at 14:52)</a>:</h4>
<p>Maybe the way to think of name/lowering/expansion is as an atomic unit for now -- it's hard enough for us to wrap our head around things</p>



<a name="158981495"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981495" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981495">(Feb 20 2019 at 14:52)</a>:</h4>
<p>really, atomic unit?</p>



<a name="158981496"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981496" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981496">(Feb 20 2019 at 14:52)</a>:</h4>
<p>and once we have some idea of that, think about how to make it incrementally re-execute?</p>



<a name="158981500"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981500" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981500">(Feb 20 2019 at 14:52)</a>:</h4>
<p>I was about to say</p>



<a name="158981504"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981504" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981504">(Feb 20 2019 at 14:52)</a>:</h4>
<p>that in terms of IDE design</p>



<a name="158981516"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981516" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981516">(Feb 20 2019 at 14:52)</a>:</h4>
<p>we may want to build in the assumption that the UX is going to interact with a tree that is not yet fully expanded</p>



<a name="158981523"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981523" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981523">(Feb 20 2019 at 14:52)</a>:</h4>
<blockquote>
<blockquote>
<p>are macros hygienic?</p>
</blockquote>
<p>Today, the answer is "sort of" -- not at the item level, in particular. But maybe worth asking why you're asking =)</p>
</blockquote>
<p>does name resolution of macro names have to take hygiene into account or can it be something really simple</p>



<a name="158981544"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981544" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981544">(Feb 20 2019 at 14:52)</a>:</h4>
<p>but no need to go into this in detail now..</p>



<a name="158981560"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981560" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981560">(Feb 20 2019 at 14:53)</a>:</h4>
<p>yeah, it seems like we're getting a bit astray somehow.</p>



<a name="158981597"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981597" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981597">(Feb 20 2019 at 14:53)</a>:</h4>
<p>I guess my feeling <span class="user-mention" data-user-id="133169">@matklad</span> is that if Rosyln / Swift are taking an approach, we'd do well to follow suit -- or at least have strong reasons to deviate.</p>



<a name="158981620"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981620" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981620">(Feb 20 2019 at 14:53)</a>:</h4>
<p>but I'm not 100% sure what that means :)</p>



<a name="158981701"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981701" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981701">(Feb 20 2019 at 14:54)</a>:</h4>
<p>Yeah, I feel the same, that's why I kinda started the meeting in the "explore a particular solution" mood :)</p>



<a name="158981719"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981719" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981719">(Feb 20 2019 at 14:54)</a>:</h4>
<p>heh :)</p>



<a name="158981722"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981722" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981722">(Feb 20 2019 at 14:54)</a>:</h4>
<p>Does Swift have token-based macros as well?</p>



<a name="158981725"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981725" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981725">(Feb 20 2019 at 14:54)</a>:</h4>
<p>I remember <span class="user-mention" data-user-id="132906">@Jonathan Turner</span> mentioning that for Typescript (and maybe C#) macros were not supported in order to make IDE support doable</p>



<a name="158981743"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981743" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981743">(Feb 20 2019 at 14:54)</a>:</h4>
<p>so doing somthing becuase roslyn does it might not be a good idea because they have different constraints</p>



<a name="158981776"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981776" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981776">(Feb 20 2019 at 14:55)</a>:</h4>
<p>(I suppose an ongoing macro expansion could perhaps be lumped in with user edits ... as in, whatever the <del>code</del> IDE+parser-library does to deal with user edits, it might also handle the changes injected by macro expansion in the same way ... and therefore we should not need to design for macros up front....)</p>



<a name="158981800"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981800" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981800">(Feb 20 2019 at 14:55)</a>:</h4>
<p>Yeah, "can we macro expand to roslyn tree" is a core question whihc might kill the design</p>



<a name="158981807"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981807" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981807">(Feb 20 2019 at 14:55)</a>:</h4>
<p>Ideally I'd like us to support everything that rustc does - I know that it was very annoying not being able to accurately find definitions introduced by C-style macros in C++</p>



<a name="158981826"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981826" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981826">(Feb 20 2019 at 14:55)</a>:</h4>
<p>So, stepping back, it feels to me like we should be doing more <em>reviewing</em> of what you've built in rust-analyzer <span class="user-mention" data-user-id="133169">@matklad</span>. I know you producd that video and other content. I wonder if it would be useful to schedule more "interactive" discussion of this kind?</p>



<a name="158981898"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981898" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981898">(Feb 20 2019 at 14:56)</a>:</h4>
<p>Basically I think we don't share quite enough context at the moment</p>



<a name="158981905"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981905" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981905">(Feb 20 2019 at 14:56)</a>:</h4>
<p>Doing the "homework" above would be prerequisite :D</p>



<a name="158981910"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981910" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981910">(Feb 20 2019 at 14:56)</a>:</h4>
<p>Or maybe it's just me :)</p>



<a name="158981929"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981929" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981929">(Feb 20 2019 at 14:56)</a>:</h4>
<p>Fair :)</p>



<a name="158981960"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981960" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981960">(Feb 20 2019 at 14:57)</a>:</h4>
<p>Maybe we need to form some homework study groups...</p>



<a name="158981975"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981975" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981975">(Feb 20 2019 at 14:57)</a>:</h4>
<p>...also, I can just step back, as I'm pretty busy right now</p>



<a name="158981981"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158981981" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158981981">(Feb 20 2019 at 14:57)</a>:</h4>
<p>you can pair up with <span class="user-mention" data-user-id="116083">@pnkfelix</span> :)</p>



<a name="158982102"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982102" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982102">(Feb 20 2019 at 14:58)</a>:</h4>
<p>So looks like it's time to wrap-up the meeting!</p>



<a name="158982144"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982144" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982144">(Feb 20 2019 at 14:58)</a>:</h4>
<p>What do you think are good next steps?</p>



<a name="158982192"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982192" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982192">(Feb 20 2019 at 14:59)</a>:</h4>
<p>1) to write down a more specific list of constaints for design</p>



<a name="158982327"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982327" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982327">(Feb 20 2019 at 15:00)</a>:</h4>
<p>2) study rosly approach in more detail to figure out if it is a good fit. Specifically, macro &amp; hygiene might be tricky</p>



<a name="158982395"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982395" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982395">(Feb 20 2019 at 15:00)</a>:</h4>
<p>3) maaaby explore alternative set ups when the tree itself is salsa-based? I kinda belive in rosly trees though</p>



<a name="158982403"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982403" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982403">(Feb 20 2019 at 15:00)</a>:</h4>
<p>So I really like the idea of "pairing up" in some sense</p>



<a name="158982436"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982436" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982436">(Feb 20 2019 at 15:01)</a>:</h4>
<p>I feel like I would like to delve more deeply but I'm trying to think about what will make it actually happen</p>



<a name="158982443"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982443" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982443">(Feb 20 2019 at 15:01)</a>:</h4>
<p>certainly scheduling some time helps :)</p>



<a name="158982490"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982490" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982490">(Feb 20 2019 at 15:01)</a>:</h4>
<p>I guess I'd be game to e.g. schedule an hour to sit down with someone (or a few people) and try to walk through what happens when (say) something is parsed in rust-analyzer?</p>



<a name="158982554"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982554" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982554">(Feb 20 2019 at 15:02)</a>:</h4>
<p>I'm mostly thinking about the "homework" phenomenon and ways to help it get done</p>



<a name="158982558"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982558" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982558">(Feb 20 2019 at 15:02)</a>:</h4>
<p>fwiw, I have very little time to actually spend on this. at least in the first half of the year :(</p>



<a name="158982580"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982580" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982580">(Feb 20 2019 at 15:02)</a>:</h4>
<blockquote>
<p>3) maaaby explore alternative set ups when the tree itself is salsa-based? I kinda belive in rosly trees though</p>
</blockquote>
<p>I feel like this is maybe not important -- in particular, I think that we'll need some "root parse" action</p>



<a name="158982584"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982584" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982584">(Feb 20 2019 at 15:02)</a>:</h4>
<p>I can totally do this! But it's better be "a few people", I do want to improve bus-factor here</p>



<a name="158982643"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982643" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982643">(Feb 20 2019 at 15:03)</a>:</h4>
<p>I plan to spend time on this</p>



<a name="158982646"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982646" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> davidtwco <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982646">(Feb 20 2019 at 15:03)</a>:</h4>
<p>I've got time I could sink into this wg if there's a desire for some more regular hands.</p>



<a name="158982685"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982685" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> davidtwco <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982685">(Feb 20 2019 at 15:03)</a>:</h4>
<p>I've not followed discussions closely so far but I can catch up.</p>



<a name="158982804"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982804" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982804">(Feb 20 2019 at 15:05)</a>:</h4>
<p>So I schedule a sync video meeting for me, <span class="user-mention" data-user-id="116009">@nikomatsakis</span>, <span class="user-mention" data-user-id="116083">@pnkfelix</span>  and <span class="user-mention" data-user-id="116107">@davidtwco</span> with the goal of diving in into the current impl</p>



<a name="158982837"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982837" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982837">(Feb 20 2019 at 15:05)</a>:</h4>
<p>I'd also love to participate, if you don't mind</p>



<a name="158982841"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982841" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982841">(Feb 20 2019 at 15:05)</a>:</h4>
<p>Seems good -- I don't think I'll have much time for writing code just now so I'd put myself as lower priority, but I'm definitely game to listen in and kibbitz</p>



<a name="158982844"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982844" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982844">(Feb 20 2019 at 15:05)</a>:</h4>
<p>anyone else is welcome to join of course!</p>



<a name="158982847"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982847" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982847">(Feb 20 2019 at 15:05)</a>:</h4>
<p>Wasn't sure what we were voting for :)</p>



<a name="158982853"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982853" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982853">(Feb 20 2019 at 15:05)</a>:</h4>
<p>Great!</p>



<a name="158982865"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982865" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982865">(Feb 20 2019 at 15:05)</a>:</h4>
<p>I am ready to write the code! I just need to know that to write :D</p>



<a name="158982928"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982928" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> nikomatsakis <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982928">(Feb 20 2019 at 15:06)</a>:</h4>
<p>I'll also try to schedule time to actually watch after the fact otherwise -- I know <span class="user-mention" data-user-id="133169">@matklad</span> you also made that other video, which I just realized I should prob just put time on my calendar for (I found it hard to find a slot otherwise)</p>



<a name="158982930"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982930" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982930">(Feb 20 2019 at 15:06)</a>:</h4>
<p>Yeah, I think fleshing out the design and current problems with our approach is the trickiest part of it all :(</p>



<a name="158982962"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982962" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982962">(Feb 20 2019 at 15:07)</a>:</h4>
<p>Ok, I think the meeting is officially over than!</p>



<a name="158982974"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982974" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982974">(Feb 20 2019 at 15:07)</a>:</h4>
<p>Gotta run, off to do more homework on the topic</p>



<a name="158982977"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158982977" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Igor Matuszewski <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158982977">(Feb 20 2019 at 15:07)</a>:</h4>
<p>Thanks!</p>



<a name="158983020"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158983020" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> pnkfelix <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158983020">(Feb 20 2019 at 15:07)</a>:</h4>
<p>thanks for setting this up <span class="user-mention" data-user-id="133169">@matklad</span> and putting up with those of us who are playing catch up</p>



<a name="158983024"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158983024" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> mw <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158983024">(Feb 20 2019 at 15:07)</a>:</h4>
<p><span aria-label="wave" class="emoji emoji-1f44b" role="img" title="wave">:wave:</span> Thanks!</p>



<a name="158990735"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158990735" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158990735">(Feb 20 2019 at 16:33)</a>:</h4>
<p><span class="user-group-mention" data-user-group-id="1060">@WG-rls2.0</span> stream/poll for rust-analyzer syntax/parsing code walkthrough: <a href="#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0/topic/rust-analyzer.20syntax.20trees.20deep.20dive" title="#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0/topic/rust-analyzer.20syntax.20trees.20deep.20dive">https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Fwg-rls-2.2E0/topic/rust-analyzer.20syntax.20trees.20deep.20dive</a></p>



<a name="158993366"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158993366" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158993366">(Feb 20 2019 at 17:07)</a>:</h4>
<p>Notes: <a href="https://github.com/rust-lang/compiler-team/pull/6" target="_blank" title="https://github.com/rust-lang/compiler-team/pull/6">https://github.com/rust-lang/compiler-team/pull/6</a></p>



<a name="158996252"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Design%20meeting%3A%20syntax%20trees/near/158996252" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Design.20meeting.3A.20syntax.20trees.html#158996252">(Feb 20 2019 at 17:44)</a>:</h4>
<p>Filed <a href="https://github.com/rust-analyzer/rust-analyzer/issues/862" target="_blank" title="https://github.com/rust-analyzer/rust-analyzer/issues/862">rust-analyzer#862</a> with list of the requirements.  It's called <code>"Finalize" design of syntax trees</code> and I hope to close it "soon-ish".</p>
<p>With syntax trees in place, we can do some bold experiments, like porting rustfmt to them (which would fix an annoyng "can't format if there are syntax errors" issue) or producing rustc AST out of them.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>